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> 10 times faster 
>» 50 times cheaper 


Blumenstock et al. (2015), Figure 3 
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Mullainathan and Spiess (2017): 
http://dx.doi.org/10.1257/jep.31.2.87 


Fragile Families Challenge 
Matthew Salganik, lan Lundberg, Alex Kindel, Sara McLanahan, 
and the participants in the Fragile Families Challenge 
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Theories focus on the predictable component, but 
empirically the unpredictable component dominates 
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» Birth cohort panel study 


> = 5,000 children born in 20 U.S. cities with an over-sample of 
non-marital births 


> Followed from birth through age 15 
» Already used in hundreds of papers and dozens of dissertations 
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Background information 
about child and family Leaderboard 


Outcomes 
» Child: GPA (continuous), Grit (continuous) 
» Household: Eviction (binary), Material hardship (continuous) 
> Primary care giver: Job training (binary), Job loss (binary) 


459 researchers applied to participate. Many worked in 
interdisciplinary teams. Goal: Make a prediction that minimizes 
mean sguare error on the hold-out set 
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More on privacy and ethics audit: 
https://arxiv.org/abs/1809.00103 
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Is this even better than a benchmark model? 
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What can we learn looking at the all the 
predictions? 
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Mullainathan and Spiess (2017): 
http://dx.doi.org/10.1257/jep.31.2.87 
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> Read: http://www.bitbybitbook.com 


> Teach: http: //www.bitbybitbook.com/en/teaching/ 
(and Italian version coming soon from il Mulino) 
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outcomes measured when child was 15 

>» outcomes are at a relatively narrow point in time rather than 
average over a longer time period (e.g., grades last semester 
vs grades in high school) 
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Dozens already happening all over the world with interesting 
similarities and differences 


Already strong community around each survey 


Code from a single Challenge can be repurposed to create 
many simulated Challenges 


Data collected with informed consent under well-developed 
ethical frameworks 


Likely to spur useful scientific developments 
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